Skip to content

Conversation

@mmtv-qa
Copy link

@mmtv-qa mmtv-qa commented Sep 15, 2025

Problem: We are running tests on multiple shards (up to 12) and want to try globalCache to speed up the pipeline. The proposed setup/teardown option does not suit us, as we want the data to be shared between all instances.

Solution: run the global cache as a standalone server.

  1. Provided the health endpoint
  2. Provided the start-server-standalone.ts script
  3. Provided the Dockerfile
  4. Provided the workflow to upload docker image (copied from my solution in https://github.com/CyborgTests/playwright-reports-server/blob/main/.github/workflows/release.yml) => need to setup credentials + test it

Problem: We are running tests on multiple shards (up to 12) and want to try globalCache to speed up the pipeline. The proposed setup/teardown option does not suit us, as we want the data to be shared between all instances.

Solution: run the cache as a separate server during test execution and “kill” it after the run.
@mmtv-qa
Copy link
Author

mmtv-qa commented Sep 15, 2025

@vitalets pls take a look

@vitalets
Copy link
Owner

Hey @mmtv-qa! Thanks for the pr, sharing data across shards is a definitely required feature.
I’ve outlined a high-level vision in #1, could you also take a look?

My thought is: let’s finalize the requirements first, and then dive into the implementation review.
A few questions about your setup:

  1. Do you plan to share data only between shards in a single run, or also across separate CI runs?
  2. Will you have parallel test runs that use the global cache?
  3. How will you run the dockerized global-cache server: will it be a single instance or scalable multi-instance app? (If the latter, we’ll need to think more carefully about the storage)

@mmtv-qa
Copy link
Author

mmtv-qa commented Sep 20, 2025

Hey @mmtv-qa! Thanks for the pr, sharing data across shards is a definitely required feature. I’ve outlined a high-level vision in #1, could you also take a look?

I have checked it. Good concept

  1. Do you plan to share data only between shards in a single run, or also across separate CI runs?
    share data only between shards in a single run
  1. Will you have parallel test runs that use the global cache?

I see this flow as follows: We launch one server within one test run, run the tests, and then shut down the server. This seems to me to be the simplest and most effective approach.

btw Our largest test run is ±600 tests, divided into 12 shards with 2 workers each.

  1. How will you run the dockerized global-cache server: will it be a single instance or scalable multi-instance app? (If the latter, we’ll need to think more carefully about the storage)
    Single instance. One dedicated instance for run

P.S. I could consider the multi-instance option, but I don't see any obvious advantages at the moment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants